UE4Sim: A Photo-Realistic Simulator for Computer Vision Applications

摘要

We present a photo-realistic training and evaluation simulator (UE4Sim) withextensive applications across various fields of computer vision. Built on topof the Unreal Engine, the simulator integrates full featured physics basedcars, unmanned aerial vehicles (UAVs), and animated human actors in diverseurban and suburban 3D environments. We demonstrate the versatility of thesimulator with two case studies: autonomous UAV-based tracking of movingobjects and autonomous driving using supervised learning. The simulator fullyintegrates both several state-of-the-art tracking algorithms with a benchmarkevaluation tool and a deep neural network (DNN) architecture for trainingvehicles to drive autonomously. It generates synthetic photo-realistic datasetswith automatic ground truth annotations to easily extend existing real-worlddatasets and provides extensive synthetic data variety through its ability toreconfigure synthetic worlds on the fly using an automatic world generationtool.